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Bottom  Line  Up  Front 


U.S.  AIR  FORCE 


■  Efforts  to  integrate  and  institutionalize  ESOH  considerations  into 
Acquisition  Systems  Engineering  continue 

■  Policy  and  guidance  in  place 

■  Focus  is  now  on  implementation  and  accountability 

■  Today’s  presentation  addresses  accountability  -  integrating 
ESOH  into  Acquisition  program  Technical  Sufficiency  Reviews 


“ Technical  Sufficiency  Reviews”  -  generic  term  for  the 
subset  of  Acquisition  program  reviews  that  assess 
technical  planning  and  management 
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Overview 
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■  Policy  evolution 

■  Four  Accountability  Tools 

■  Programmatic  Environment,  Safety,  and  Occupational  Health 
Evaluation  (PESHE) 

■  System  Safety  -  ESOH  Management  Evaluation  Criteria  for 
DoD  Acquisition 

■  Program  reviews  of  High  and  Serious  ESOH  risks 

■  Program  Support  Review  (PSR) 

■  Integrating  ESOH  into  Systems  Engineering  Booklet 
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Policy  Evolution 


U.S.  AIR  FORCE 


■  Initial  focus  on  integrating  ESOH  into  Acquisition  systems 
engineering  policy  and  guidance 

■  DoDI  5000.02  (8  Dec  08) 

■  Evolution  began  in  1996 

■  Incorporates  key  policy  memos  from  2004-2007 

■  Defense  Acquisition  Guidebook  (DAG) 

■  Defines  expectations  for  how  to  implement  DoDI  5000.02 

■  Provides  detailed  guidance 

■  “System  Safety  in  Systems  Engineering”  DAU  CLE009 

■  Established  in  2005 

■  Over  2000  graduates 
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Policy  Evolution  (cont’d) 
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■  Subsequent  focus  on  revitalizing  systems  engineering 

■  DoD  emphasis  began  in  2003 

■  ESOH,  as  part  of  systems  engineering,  also  benefited 

■  Reflected  in  DoDI  5000.02  and  DAG 

■  Re-established  OSD  and  Service  oversight  of  programs 

■  Current  focus  of  both  systems  engineering  and  ESOH  on 
accountability  by  mandating  reviews 

■  PESHE 

■  System  Safety  -  ESOH  Management  Evaluation  Criteria  for 
DoD  Acquisition 

■  Program  reviews  of  High  and  Serious  ESOH  risks 

■  PSR 
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U.S.  AIR  FORCE 


■  Mandated  by  DoDI  5000.02  (dating  back  to  1996) 

■  Required  contents 

■  Describes  the  strategy  for  integrating  ESOH  into  systems 
engineering 

■  Documents  the  status  of  program  ESOH  risk  management 

■  Required  reviews 

■  Milestone  B,  Milestone  C,  and  Full  Rate  Production  Decisions 

■  In  support  of  Systems  Engineering  Plan  (SEP)  and 
Acquisition  Strategy  reviews 

■  E2S2  presentations  on  Monday  and  Thursday 

Essential  for  PESHE  to  Document  Actual  Program  Office 

Activities ,  not  regurgitate  policy 
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System  Safety  -  ESOH  Management 
Evaluation  Criteria  for  DoD  Acquisition 


■  Tool  to  assess  SE  technical  discipline  in  the 
integration  of  ESOH  using  System  Safety 
methodology 

■  Technical  and  Program  Reviews 
(self  assessment) 

■  Milestone  Review  Process 
(oversight  assessment) 

■  Four  key  areas  for  evaluation 

■  Planning 

■  Requirements  Analysis 

■  Hazard  analysis 

■  Resources 

■  Assessment  criteria  for  each  area  for  each  life 
cycle  phase 

■  Weighted  summation  of  four  ratings  to  overall 
rating  for  each  life  cycle  phase 


SYSTEM  SAFETY  -  ESOH 
MANAGEMENT  EVALUATION 
CRITERIA 

FOR  DOD  ACQUISITION 


(Version  1.1) 


January  1007 
Departmtnf  of  Defence 
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Program  reviews  of  High  and 

Serious  ESOH  risks 


■  2004  USD(AT&L)  memo  required  reporting  of  ESOH  risk  status  at 
technical  and  program  reviews  (as  appropriate) 

■  2006  USD(AT&L)  memo  required  reporting  of  High  and  Serious 
risks  at  all  program  reviews  (internal  and  external) 

■  Incorporated  into  DoDI  5000.02 

■  Purpose: 

■  Provides  periodic  reassessments  of  risk  acceptance 
decisions 

■  Maintains  visibility  on  efforts  to  mitigate  risks 

■  Requirements: 

■  Report  on  all  “current”  high  and  serious  risks  regardless  of 
acceptance  status 

■  Document  details  in  PESHE 
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Program  Support  Review  (PSR) 


■  OSD  began  in  2003 

■  DoDI  5000.02  (2008)  requirements 

■  “PSRs  are  a  means  to  inform  an  MDA  and 
Program  Office  of  the  status  of  technical 
planning  and  management  processes  by 
identifying  cost,  schedule,  and  performance 
risk  and  recommendations  to  mitigate  those 
risks” 

■  “PSRs  shall  be  conducted  by  cross¬ 
functional  and  cross-organizational  teams 
appropriate  to  the  program  and  situation” 

■  Teams  use  Defense  Acquisition  Program  Support 
(DAPS)  Methodology 

■  Section  4.0  Technical  Process 

■  Sub  Area  4.1  Design  Considerations 

■  Factor  4.1.4  ESOH 


Systems  and  Software  Engineering 

DEFENSE  ACQUISITION  PROGRAM  SUPPORT 

Methodology 

Version  2.0 


Office  of  the  Deputy  Under  Secretary  of  Defense 
for  Acquisition  and  Technology 

Systems  and  Software  Engineering 
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Program  Support  Review  (PSR) 


Mission  Capabilities 
Concept  of  Operations 
Mission  Description 
Family  of  Systems/System 
of  Systems 

Dependencies/Interfaces 
Analysis  of  Alternatives 
Validity  and  Currency 
Linkage  and  Traceability 
Capabilities 

Reasonableness,  Stability, 
and  Testability 
Key  Performance 
Parameters  and  Key 
System  Attributes 
Resources 
Program  Schedule 
Overview  (Tier  1) 
Viability 

Constraints  and 
Dependencies 
Budget  Sufficiency  and 
Phasing 

Program  Funding  and 
Allocation 
Program  Schedule 

Overview  (Tier  1 ) 


Viability 

Constraints  and 
Dependencies 
Budget  Sufficiency  and 
Phasing 

Program  Funding  and 
Allocation 

Continuity  and  Stability 
Staffing  Level 
Sufficiency  of  Numbers 
and  Qualifications 
Continuity  and  Stability 
Management 
Acquisition  Strategy 
Credibility 
Acceptability 
Knowledge-Based 
Decisions  and  Milestones 
Statutory  and  Regulatory 
Compliance  and  Guidance 
Entrance  and 
Exit/Success  Criteria 
Certifications 
Program  and  Project 
Management 
Program  Plan/Schedule 
(SMP/IMS) 


Work  Breakdown 
Structure  (WBS) 
Management  Structure 
and  Communications 
Management  Methods, 
Metrics,  and  Techniques 
Information  Management 
(IDE,  IT,  Data  Rights) 
Management  of 
Dependencies  and 
External  Interfaces  (FoS 
SoS) 

Contracting 
Prime  Contractor 
Management 
Subcontractor 
Management 
Value  Engineering 
Technical  Process 
Design  Considerations 
System  Assurance 
Modular  Open  Systems 
Approach 
Architecture 
Corrosion 

Spectrum  Management 
Sustainment  as  a  Design 
Consideration 


Environment,  Safety, 
and  Occupational  Health 

Human  Systems 
Integration  (HSI) 
Requirements 
Development 
Analysis  and 
Decomposition 
Management  of 
Requirements 
/  Technology  Maturity  and 
Integration 
Technical  Baselines 
Technical  Review  Planning 
Configuration 
Management 
Baseline  Stability 
Engineering  Tools 
Systems  and  Software 
Engineering  (SSE)  Tools 
Modeling  and  Simulation 
(M&S)  Tools 
Producibility  and 
Production  Planning  Tools 
Software 

Software  Development 
Plan 
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Estimation 
Design  Verification 
Test  and  Evaluation  Plan 
Verification  Correlation 
Supportability  Planning 
Acquisition  Logistics 
Performance-Based 
Logistics  (PBL) 
Sustainment 
Performance 
Effectiveness 
Design  Capabilities 
Assessment 
Suitability 

Reliability  Assessment 
Availability  Assessment 
Maintainability 
Assessment 
Survivability 
Live  Fire  Test  and 
Evaluation  Assessment 
Production 

Assessed  Manufacturing 
Assessed  Quality 
Special  Interest  Areas 
Readiness  Levels 
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Integrating  ESOH  Into  Systems 

Engineering  Booklet 


Integrating  ESOH  into  Systems  Engineering 


Builds  on  CLE009 
and  depicts  when 
ESOH  activities 
should  be 
performed  to 
influence  system 
design  throughout 
the  systems 
engineering 
process 

System  Safety- 
ESOH  Mgt. 
Evaluation  Criteria 
are  included 
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Summary 
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■  Efforts  to  integrate  and  institutionalize  ESOH  considerations  into 
Acquisition  Systems  Engineering  continues 

■  Policy  and  guidance  in  place 

■  Focus  is  now  on  implementation  and  accountability 

■  Today’s  presentation  addresses  accountability  -  integrating 
ESOH  into  Acquisition  program  Technical  Sufficiency  Reviews 


“Technical  Sufficiency  Reviews”  -  generic  term  for  the 
subset  of  Acquisition  program  reviews  that  assess 
technical  planning  and  management 
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Questions? 


w 
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